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Abstract 

Given  a  program  that  has  access  to  some  private  information,  how  can  we  ensure 
that  it  does  not  improperly  leak  the  information?  We  formalize  the  desired  security 
property  as  a  property  called  noninterference.  We  discuss  versions  of  noninterference 
appropriate  for  multi-threaded  programs  with  probabilistic  scheduling  and  describe 
rules  for  ensuring  noninterference. 


1  Introduction 

Ensuring  the  privacy  of  information  is  a  major  problem  today,  made  both 
more  pressing  and  more  difficult  by  the  enormous  growth  of  the  Internet.  In 
this  paper,  we  address  one  aspect  of  this  problem:  given  a  program  P  that  has 
access  to  some  private  information,  how  can  we  prevent  P  from  leaking  the 
information?  (This  problem  was  called  the  confinement  problem  by  Lampson 
[6],  who  first  raised  the  issue  in  the  early  1970s.)  We  will  focus  in  particular 
on  the  case  when  P  is  multi-threaded. 

The  difficulty  of  preventing  a  program  P  from  leaking  private  information 
depends  greatly  on  what  kinds  of  observations  of  P  are  possible.  If  we  can 
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tax  applet 


Fig.  1.  A  Tax  Return  Applet 


make  external  observations  of  P’s  running  time,  memory  usage,  and  so  forth, 
then  preventing  leaks  becomes  very  difficult.  For  example,  P  could  modulate 
its  running  time  in  order  to  encode  the  private  information.  Furthermore, 
these  modulations  might  depend  on  low-level  implementation  details,  such  as 
paging  and  caching  behavior.  But  this  means  that  it  is  insufficient  to  prove 
confinement  with  respect  to  an  abstract  semantics — every  implementation  de¬ 
tail  that  affects  running  time  must  be  addressed  in  the  proof  of  confinement. 
For  this  reason,  we  will  not  consider  such  external  observations  further. 

If,  instead,  we  can  only  make  internal  observations  of  P’s  behavior,  the 
confinement  problem  becomes  more  tractable.  Internal  observations  include 
the  values  of  program  variables,  together  with  any  system-provided  functions 
that  can  be  called  by  P.  Of  course,  if  the  system  provides  a  real-time  clock, 
then  running  time  is  observable  internally,  and  we  are  no  better  off  than  before. 
But  in  this  case  we  can  design  the  system  with  confinement  in  mind,  excluding 
features  (like  real-time  clocks)  that  are  problematic.  This  situation  is  relevant 
to  the  case  of  mobile  code,  which  runs  under  the  control  of  a  host  machine 
that  can  limit  what  the  code  can  observe. 

When  only  internal  observations  are  possible,  we  can  formulate  the  con¬ 
finement  problem  as  follows  [1,2]:  if  each  program  variable  is  classified  as  L 
(low,  public)  or  H  (high,  private),  then  we  wish  to  ensure  that  information 
cannot  flow  from  H  variables  to  L  variables. 

For  example.  Figure  1  suggests  the  behavior  of  a  tax  return  applet  which 
could  be  downloaded  from  a  site  called  TrustMe.  The  applet  runs  on  my 
machine,  allowing  me  to  complete  my  tax  return.  When  I  finish,  the  applet 
sends  the  completed  tax  return  to  the  IRS  and  sends  billing  information  back 
to  TrustMe,  using  encryption  to  protect  the  privacy  of  these  communications. 
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But  how  do  I  know  that  my  private  financial  information  is  not  somehow 
encoded  in  the  billing  information  sent  back  to  TrustMe?  If  I  classify  the  tax 
return  as  H  and  the  billing  information  as  L,  then  I  would  like  to  know  that 
no  information  can  fiow  from  H  variables  to  L  variables. 

2  Possibilistic  Noninterference 

Formally,  we  want  programs  to  satisfy  a  property  called  noninterference  [10], 
which  says  that  the  final  values  of  L  variables  don’t  depend  on  the  initial 
values  of  H  variables.  In  the  case  when  programs  are  multi-threaded,  and 
hence  nondeterministic,  we  need  a  possibilistic  noninterference  property  [8], 
which  says  that  changing  the  initial  values  of  H  variables  cannot  change  the 
set  of  possible  final  values  of  L  variables. 

Here’s  a  non-example,  similar  to  one  in  [8].  Suppose  x  is  H,  with  value 
0  or  1,  and  y  is  L.  Also,  assume  that  t  is  initially  0.  Consider  the  following 
program,  which  consists  of  two  threads: 

Thread  a:  if  X  =  1  then 

while  t  =  0  do  skip; 

y  ■=  1; 

t  :=  1 

Thread  (3:  if  a:  =  0  then 

while  t  =  0  do  skip; 

y  ■=  0; 

t  :=  1 

Note  that  thread  a  always  assigns  1  to  y,  and  thread  (3  always  assigns  0  to  y, 
but  the  order  in  which  these  assignments  are  done  depends  on  the  value  of  x. 
As  a  result,  with  any  fair  scheduler  the  value  of  x  is  copied  to  y. 

Suppose  that  we  adopt  a  formal  semantics  for  our  multi-threaded  language 
that  specifies  a  purely  nondeterministic  scheduler.  Such  a  scheduler  is  charac¬ 
terized  by  the  simple  rule: 

At  each  step,  any  thread  can  be  selected  to  run  for  a  step. 

Suppose  that  we  prove  that  a  program  P  satisfies  possibilistic  noninterference 
with  respect  to  this  scheduler.  Can  we  conclude  that  P  remains  secure  if  we 
implement  something  more  deterministic,  such  as  round-robin  time  slicing? 

The  answer  is  no.  For  suppose  that  x  is  H,  with  value  0  or  1,  y  is  L,  and 
c  is  a  command  that  doesn’t  alter  x  or  y,  but  that  takes  longer  than  a  time 
slice.  Consider  the  following  program: 

Thread  a:  if  X  =  1  then  (c;  c); 

y  ■=  1 

Thread  f3:  c; 

y  :=0 
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With  respect  to  the  purely  nondeterministic  scheduler,  this  program  satisfies 
possibilistic  noninterference:  regardless  of  the  initial  value  of  x,  the  final  value 
of  y  can  be  either  0  or  1.  But  under  round-robin  time  slicing,  the  value  of 
X  is  always  copied  to  y.  Thus  we  see  that  noninterference  is  not  a  safety 
property — it  is  not  closed  under  trace  subsetting. 


3  Probabilistic  Noninterference 

A  purely  nondeterministic  scheduler  is  convenient  in  a  formal  semantics,  but 
it  is  unclear  how  such  a  scheduler  might  be  implemented;  it  seems  to  require 
an  “erratic  daemon” .  ^ 

We  might  consider  a  probabilistic  implementation  that  flips  coins  to  select 
the  thread  to  run  in  the  next  step.  But  note  that  this  moves  us  from  a  nonde¬ 
terministic  semantics,  in  which  events  are  either  possible  or  impossible,  to  a 
probabilistic  semantics,  in  which  events  have  a  probability  of  occurring.  Still, 
we  can  say  that  this  gives  an  implementation  of  the  purely  nondeterministic 
scheduler,  if  we  are  willing  to  equate  “possible”  with  “occurs  with  nonzero 
probability” . 

But  now  suppose  that  x  is  H,  with  value  between  1  and  100,  and  y  is 
L.  Suppose  that  random{100)  returns  a  random  number  between  1  and  100. 
Consider  the  following  program: 

Thread  a:  y  :=  x 

Thread  fd:  y  :=  randomilQQ) 

This  program  satisfies  possibilistic  noninterference:  regardless  of  the  initial 
value  of  X,  the  final  value  of  y  can  be  any  number  between  1  and  100.  But 
with  a  probabilistic  semantics,  this  is  not  good  enough,  because  the  final  values 
of  y  are  not  equally  likely.  In  particular,  if  we  can  run  the  program  repeatedly, 
we  expect  the  final  values  of  y  to  look  something  like 

75,22,12,22,22,93,4,  22,... 

allowing  us  to  conclude  (in  this  case)  that  x  is  probably  22.  Thus  we  see  that 
possibilistic  noninterference  is  not  sufficient  to  prevent  probabilistic  informa¬ 
tion  flows.  ^  Instead,  we  now  need  a  probabilistic  noninterference  property, 
which  says  that  changing  the  initial  values  of  H  variables  cannot  change  the 
joint  distribution  of  possible  final  values  of  L  variables  [9].  In  the  next  section, 
we  develop  this  idea  more  formally. 


^  The  term  is  due  to  Dijkstra  [3]. 

^  This  observation  can  be  credited  to  McLean  [7]  and  Wittbold  and  Johnson  [11]. 
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4  Multi-Threaded  Programs  as  Markov  Chains 

We  assume  that  threads  are  written  in  a  simple  imperative  language: 

c  ::=  skip 
I  X  :=  e 

I  Cl ;  C2 

I  if  e  then  ci  else  C2 
I  while  e  do  c 

Integers  are  the  only  values;  we  use  0  for  false  and  nonzero  for  true.  We  assume 
that  all  expressions  are  pure  and  total,  and  that  expressions  are  executed 
atomically. 

Programs  are  executed  with  respect  to  a  single  shared  memory  //,  which  is 
a  map  from  identifiers  to  integers.  We  extend  this  to  a  map  from  expressions 
to  integers,  writing  //(e)  to  denote  the  value  of  expression  e  in  memory  //. 

The  semantics  of  commands  is  given  by  a  standard  transition  semantics 
— on  configurations  (c,  //)  or  //.  The  rules  are  given  in  Figure  2. 

A  multi-threaded  program  is  modeled  by  an  object  map  O  that  maps 
thread  identifiers  (a,  /3,  . . . )  to  commands.  The  semantics  of  multi-threaded 
programs  is  given  via  global  transitions  on  global  configurations  (O,//). 
The  three  rules  are 

(global)  0{a)  =  c 
{c,id) — >11' 

p  =  V|o| _ 

{0,p)^{0  -  a,  Id') 

0(a)  =  c 
(c,ii) — >(c!,ii') 

p  =  V|o| _ 

(0,ii)^(0[a  :=  c'lii') 

({},//)^({},//) 

The  first  and  second  rules  deal  with  a  nonempty  set  of  threads;  the  third  deals 
with  an  empty  set  of  threads.  Note  that  we  are  assuming  a  uniform  scheduler, 
that  selects  each  thread  in  O  with  equal  probability. 

With  these  definitions,  a  program  O  executing  in  memory  //  is  a  Markov 
chain  [4].  The  states  of  the  Markov  chain  are  all  the  global  configurations 
reachable  from  the  initial  state  (O,  //)  under  and  the  transition  matrix  T 
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(noop) 

{skip,  //) — >/i 

(update) 

X  e  dom{ii) 

(x  :=  e,  fi) — >ii[x  :=  //(e)] 

(sequence) 

(ci,//) — >11' 

(ci;C2,//) — >{c2,n') 

(ci,//) — 

(ci;C2,//) — ^(c;;C2,//') 


(branch)  //(e)  ^  0 

(if  e  then  ci  else  C2,//) — ?'(ci,//) 

//(e)  =  0 

(if  e  then  ci  else  C2,//) — >{c2, 11) 

(loop)  //(e)  =  0 

(while  e  do  c, //) — ?•// 

//(e)  nonzero 

(while  e  do  c, //) — ?-(c;  while  e  do  c, //) 
Fig.  2.  Sequential  Transition  Semantics 


is  given  by 


r((Oi,  //i),  (O2,  /^2)) 


p,  a  {Oi,  pi)^{02,  P2) 

0,  otherwise 


It  is  now  useful  to  define  a  probabilistic  state  ?/  to  be  a  (discrete)  probability 
distribution  on  the  set  of  global  configurations  [5].  Concretely,  ?/  is  a  row  vector 
with  unit  sum.  With  this  viewpoint,  we  can  model  the  execution  of  O  under 
memory  //  as  a  deterministic  sequence  of  probabilistic  states: 


?/0,  Ui,  U2,  U3,  . . . 

where  uq  is  the  distribution  that  assigns  probability  1  to  (O,  //)  and  0  to 
all  other  configurations,  and  Uk+i  =  UkT.  We  can  now  write  a  very  simple 
expression  for  the  kth  probabilistic  state:  Uk  =  UqT'^. 
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For  example,  consider  the  program 

a  :  while  /  =  0  do  skip 
(3:  l:=l 


executed  in  a  memory  that  sets  I  to  0  initially.  There  are  a  total  of  five 
reachable  states  in  the  Markov  chain: 

a  :  while  /  =  0  do  skip 

1) 

q2  :  ({o  :  while  /  =  0  do  skip},  [I  :=  1]) 
a  :  skip;  while  /  =  0  do  skip  1 

0  :  ((  :=  1)  J 

^4  :  ({q;  :  sfciip;  while  /  =  0  do  skip},  [I  :=  1]) 

:  ({  },  [/  :=  1]) 


The  transition  matrix  T  for  this  program  is  as  follows: 


Qi 

Q2 

Q3 

Qi 

Q5 

Qi 

0 

1/2 

1/2 

0 

0 

Q2 

0 

0 

0 

0 

1 

Qs 

1/2 

0 

0 

1/2 

0 

Qi 

0 

1 

0 

0 

0 

Q5 

0 

0 

0 

0 

1 

For  instance,  we  get  the  first  row  of  T  by  noting  that  running  thread  a  from 
state  Qi  takes  us  to  state  q^,  and  running  thread  (3  takes  us  to  state  q2-  Thus, 
under  our  uniform  scheduling  assumption,  we  go  from  qi  either  to  q2  or  to  qz, 
each  with  probability  1/2. 
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In  terms  of  probabilistic  states,  the  initial  distribution  Uq  is  (1  0  0  0  0). 
And  we  can  trace  the  probabilistic  states  that  O  passes  through: 


Qi 

Uo 

1 

0 

0 

0 

0 

UqT 

0 

1/2 

1/2 

0 

0 

uoT^ 

1/4 

0 

0 

1/4 

1/2 

uoT^ 

0 

3/8 

1/8 

0 

1/2 

UoT^ 

1/16 

0 

0 

1/16 

7/8 

UoT^ 

0 

3/32 

1/32 

0 

7/8 

uoT^ 

1/64 

0 

0 

1/64  31/32 

Of  course,  O  will  converge  to  the  probabilistic  state  (0  0  0  0  1). 

Now,  to  formalize  the  probabilistic  noninterference  property,  we  need  to 
define  a  notion  of  equivalence  on  probabilistic  states.  To  this  end,  we  say 
that  probabilistic  states  u  and  u'  are  equivalent,  written  u  ^  u',  if  they  are 
equal  after  H  variables  are  projected  out.  Intuitively,  u  and  u'  agree  about 
everything  except  the  values  of  H  variables.  For  example,  if  a:  is  and  y  is 
L,  then 

I  {O,[x:=0,y:=0]):l/2,] 
\{O,[x:=l,y:=0]):l/2  j 

is  equivalent  to 

{{O,[x:=2,y:=0]):l}, 

since  in  both  cases  the  result  of  projecting  out  x  is 

{{0,[y:=  0]):!}. 

Finally,  we  can  give  the  formal  definition  of  probabilistic  noninterference: 

Definition  4.1  Program  O  satisfies  probabilistic  noninterference  if  for  all 
probabilistic  states  u  and  u' ,  u  ^  u'  implies  uT  ^  u'T. 

This  definition  gives  us  what  we  want,  for  suppose  that  we  execute  a  pro¬ 
gram  O  under  two  memories  y  and  y'  that  agree  on  the  values  of  L  variables. 
Then 

and  hence 

for  all  k.  That  is,  the  two  executions  proceed  in  probabilistic  lockstep. 
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5  Ensuring  Probabilistic  Noninterference 

We  can  perform  a  static  analysis  that  ensures  that  a  program  O  satisfies 
probabilistic  noninterference.  The  analysis  is  described  formally  (as  a  type 
system)  in  [9];  here  we  give  an  intuitive  presentation  as  a  set  of  rules.  The 
rules  impose  constraints  on  assignments,  while  loops,  and  if  statements: 

•  For  an  assignment,  y  :=  e,  the  rule  is  that  if  y  is  L,  then  e  must  contain  no 
H  variables. 

•  For  a  while  loop,  while  e  do  c,  the  rule  is  that  e  must  contain  no  H 
variables. 

•  For  an  if  statement,  if  e  then  c  else  c',  the  rule  is  that  if  e  contains  any 
H  variables,  then 

(i)  c  and  c'  must  contain  no  assignments  to  L  variables, 

(ii)  c  and  c'  must  contain  no  while  loops,  and 

(iii)  the  entire  if  statement  must  be  protected,  so  that  it  executes  atomically. 

For  the  last  rule,  we  introduce  a  new  command,  protect  c,  whose  seman¬ 
tics  is  given  by 

(atomicity)  (c, //) — >*  jl' 

(protect  c, //) — >ii' 

That  is,  if  (c,  //)  can  reach  y'  in  one  or  more  steps,  then  (protect  c,  //)  can 
reach  y'  in  exactly  one  step. 

Applying  these  rules  to  the  first  program  of  Section  2,  we  see  that  the 
program  is  illegal,  because  both  threads  have  while  loops  within  the  bodies  of 
if  statements  whose  guards  are  H.  And  for  the  second  program  of  Section  2  to 
be  legal,  the  if  statement  of  thread  a  needs  to  be  protected;  this  will  mask  the 
amount  of  time  needed  to  execute  it,  thereby  eliminating  the  timing  channel. 
Of  course,  our  rules  are  necessarily  conservative.  More  experience  is  needed 
to  determine  how  burdensome  they  are  in  practice. 

It  can  be  shown  that  any  program  O  that  satisfies  the  above  rules  satisfies 
probabilistic  noninterference.  Details  can  be  found  in  [9];  here  we  sketch  part 
of  the  argument. 

First,  we  can  show  probabilistic  noninterference  for  point  masses;  that  is, 
for  distributions  in  which  some  configuration  has  probability  1  and  all  others 
have  probability  0: 

Theorem  5.1  If  O  satisfies  the  above  rules  and  l  ^  l' ,  where  l  and  l'  are 
point  masses,  then  lT  ^  l'T . 

Then  we  can  extend  the  result  to  arbitrary  distributions  by  exploiting  the 
linearity  of  T : 

Lemma  5.2  If  Ui  ^  u[,  for  all  i,  then 

(il'i/l  -\-  0,2^2  “h  0,2^2  “1“  *  *  * 
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Lemma  5.3  If  u  u' ,  then  there  exist  eoeffieients  Ci, 02,03,...  and  point 
masses  ii,  L2, 13,  ■  ■  ■  and  i[,  4?  4?  •  •  •  ^  4  sueh  that 

U  =  Oiii  +02^2  +  ^3^3  +  •  •  • 


and 


u'  —  0i4  +  024  “1“  C34  +  ■  ■  ■ 


Corollary  5.4  If  O  satisfies  the  above  rules  and  u  ^  u' ,  then  uT  ^  u'T. 


Proof.  Since  T  is  a  continuous  linear  transformation, 

uT  =  (Oiii  +  02^2  +  ^3^3  +  •  •  •)r 

=  Ci(iir)  +  02(^2^)  +  C3{l3T)  +  •  •  • 

^  ci(4r)  +  C2(4r)  +  c3{i'^T)  +  •  •  • 

=  (Oi4  +  C24  +  C3L'fi)T 
=  u'T 

□ 


6  Conclusion 

To  develop  secure  computer  systems,  it  is  first  necessary  to  identify  the  precise 
security  properties  of  interest.  We  have  presented  one  such  property,  proba¬ 
bilistic  noninterference,  aimed  at  protecting  information  privacy  and  we  have 
described  rules  sufificient  to  guarantee  it;  our  hope  is  that  such  rules  provide 
a  basis  for  constructing  provably-secure  systems  in  practice. 
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